PhDGPT: Measuring bias in academic mental health across quantised and uncensored LLMs

Abstract

We investigate bias in academic mental health as reflected by Large Language Models (LLMs). To this aim, we introduce PhDGPT, a synthetic dataset encapsulating the machine psychology of Ph.D. researchers and professors as perceived by OpenAI’s GPT-3.5 and GPT-OSS-20b. The dataset consists of 18,000 data points for each model, comprising 300 iterations repeated across 15 academic events, 2 gender personas, 2 career levels and 42 unique items responses of the Depression, Anxiety, and Stress Scale (DASS-42) across 2 LLMs. The PhDGPT dataset integrates these psychometric scores with their explanations in plain language. This synergy of scores and texts offers a dual, comprehensive perspective on..